Robust supervised learning with coordinate gradient descent

This paper considers the problem of supervised learning with linear methods when both features and labels can be corrupted, either in form heavy tailed data and/or corrupted rows. We introduce a combination coordinate gradient descent as algorithm together robust estimators partial derivatives. leads to statistical that have numerical complexity nearly identical non-robust ones based on empirical risk minimization. The main idea is simple: while requires computational cost robustly estimating whole update all parameters, parameter updated immediately using estimator single derivative descent. prove upper bounds generalization error algorithms derived from this idea, control optimization errors without strong convexity assumption risk. Finally, we propose an efficient implementation approach new Python library called linlearn, demonstrate through extensive experiments our introduces interesting compromise between robustness, performance efficiency for problem.